Solutions to the equivalent mutants problem: A systematic review and comparative experiment

Context. Mutation testing is a fault-based technique for measuring the effectiveness of a test set in terms of its ability to detect faults. Mutation testing seeds artificial faults into an application and checks whether a test suite can detect them. If these faults are not found, the test suite is still not considered to be ‘good enough’. However, there are also mutations which keep the program semantics unchanged and thus cannot be detected by any test suite. Finding a way to assess these mutations is also known as the equivalent mutant problem (EMP). Objectives. The main objective of this thesis is to conduct a systematic literature review in the field of mutation testing, to identify and classify existing methods for equivalent mutants detection. In addiction, other objectives are: analyze possibilities to improve existing methods for equivalent mutant detection, implement new or improved method and compare it with existing ones. Methods. Based on the systematic literature review method we have went over publications from six electronic databases and one conference proceedings. Standard method was extended by scanning lists of references and some alternative sources: searching in Google Scholar, checking personal websites of relevant authors and contacting all of them. We have performed all the systematic literature review steps such as the protocol development, initial selection, final selection, quality assessment, data extraction and data synthesis. In the second part of this thesis – an experiment, we have implemented four second order mutation testing strategies and compared them each other from four different perspectives: mutants reduction, equivalent mutants reduction, fault detection loss and mutation testing process time reduction.